Access line bonding and splitting methods and apparatus

ABSTRACT

Methods and apparatus for aggregating communication channels. In one exemplary embodiment an aggregation device is required only on one side of the set of communication channels being aggregated. Advantageously, no modifications to the source and/or destination of data transfers are required for operation of the invention. In one embodiment utilized for data transfers in the downstream direction, object download requests can be fulfilled by sending a sequence of partial download requests through the set of communication channels being aggregated, and forwarding the resulting partial data objects received to the original requestor in the proper order. In another embodiment utilized for upstream data transfers, traffic is bifurcated to travel along a set of communication channels, and the bifurcated traffic is reassembled either at the network layer or the transport layer.

PRIORITY

This application claims priority to U.S. Provisional Patent ApplicationSer. No. 60/901,078 filed Feb. 12, 2007 entitled “ACCESS LINE BONDINGMETHODS AND DEVICES”, which is incorporated herein by reference in itsentirety.

COPYRIGHT

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

FIELD OF THE INVENTION

The invention relates generally to the field of network communications,including Internet and intranet communications. The invention concernsin one exemplary aspect methods and devices permitting network (e.g.,Internet access lines or intranet communication links) to be aggregated,thereby enabling faster communications for applications such as Internetaccess and intranet communication.

BACKGROUND OF THE INVENTION

Dial-up service is a common method of Internet access. Dial-up serviceuses a dial-up modem through which a computer makes phone calls to anInternet service provider. The dial-up modem transforms digital datafrom the personal computer into an analog signal for transmissionthrough a phone line, and conversely converts incoming analog signalsinto digital data for the personal computer. Dial-up service is known tobe relatively slow compared to newer broadband access architectures. Forexample, viewing web pages with multimedia content, such as graphicalimages, is often unacceptably slow to many users.

Broadband access addresses these problems encountered when attempting toview, e.g. multimedia content, etc., by providing higher digital datarates than dial-up service. A “DSL” (Digital Subscriber Line) involvesupgrading the dial-up modem to a higher speed modem, known as a DSLmodem, as well as using an upgraded modem device at the Internet ServiceProvider (ISP) premises, also known as the “central office.” The DSLapproach uses existing copper wire, possibly upgraded along certainsegments to increase its capacity to carry digitized information. TheDSL data is carried over the same telephone line that is alreadyconnected to the customer location, such as a business or a residentialunit, and the central office that is owned and operated by the localtelephone company. The DSL data is carried over a different frequencyband than used for voice communications. The data rates achievable byDSL are dependent on the distance between the customer location and thecentral office, and commonly range roughly between 100 kbps-1500 kbps. ADSL connection is commonly called a broadband access line. Many DSLservice offerings in use today are asymmetric. In particular, the datarate in the downstream direction, from the central office to thecustomer, is faster than in the upstream direction. This is determinedby the relative widths of the frequency bands used in the upstream anddownstream directions on the copper wires.

Another broadband access implementation commonly used with residentialcustomers makes use of the coaxial cable that passes through aresidential unit, for purposes of providing television signals to thehome, i.e., “Cable TV”. A specialized modem, called a cable or DOCSISmodem, is attached the coaxial cable inside the residential unit. Thecable modem facilitates digital communication between the residentialunit and facilities owned by the cable TV operator (often called a“Multiple Systems Operator” (MSO)). The MSO is attached to the Internet,and thus becomes an Internet service provider for the residential unit.The coaxial cable entering a residential unit is typically shared withother residential units in close geographic proximity. Currently, peakdata rates on the order of approximately 5 to 10 million bits per second(Mbps) and beyond are commonly obtained with cable modems. As with DSL,cable Internet service is usually asymmetric in that the data rate inthe downstream directions is greater than in the upstream direction.Peak data rates available in the downstream direction with cable modemtechnology are commonly greater than that with DSL modem technology.

Small business users may subscribe to a DSL service or a cable servicefor Internet access. Internet service providers may offer a “businessclass” DSL or cable service, which provides higher reliability andavailability guarantees, as well as less aggressive statisticalmultiplexing (i.e. sharing of network capacity) in the backhaul networkfor Internet connectivity, as compared to the residential serviceofferings.

An older and more expensive transport technology, called T1 service, isalso commonly used by businesses for Internet connectivity. The datarate available on a T1 line is approximately 1.5 million bits persecond, in each direction. Though the data rate on a T1 line may be lessthan that available on a DSL line, often a business may prefer a T1service for Internet access. This is because the T1 service may connectthe user to a higher capacity backhaul network. Also, the signalstransmitted on a T1 are more robust to interference from “crosstalk”across different twisted copper pairs, which are physically adjacent.This means that the data rate of 1.5 million bits per second isavailable, regardless of signal transmissions on other twisted copperpairs. It is perceived that DSL service is more sensitive to crosstalkthan T1. The data rate used on a DSL line may vary dynamically accordingto the degree of crosstalk that exists. For these reasons, T1 servicefor Internet access may be preferred by a business over DSL, even thoughit is much more expensive. Large businesses may also use T1 lines todirectly connect remote office locations, rather than for Internetaccess.

In order to accommodate greater data rates than available with a singleInternet access service, a business may subscribe to several Internetservice providers, or may subscribe to more than one instance of thesame service from a single Internet service provider. In order tofacilitate connectivity to several ISPs or several instances of an ISPservice, a business may use a router device with a load balancingcapability. Outgoing traffic from the business may be spread across theaccess lines, at the granularity of an end user or perhaps a TCPsession. If the number of simultaneous active users in the business islarge most of the time, this may be an adequate solution. However, ifthis is not the case, many of the access lines may be idle most of thetime. For example, if a user within the business downloads a file, onlyone of the access lines is used to support the download, even though theother access lines are idle.

In order to make full use of access lines, physical or link layerbonding of access lines may be used, if the service is available.Typically, a user may only bond together access lines from the sameservice provider. DSL lines may be bonded together at the physicallayer, whereby data symbols on each access line are transmittedsynchronously with respect to each other, forming a virtual high-speedlink from different data channels. Alternatively, DSL lines can bebonded together at the link layer for a virtual high-speed link. In thismethod, a data packet of the virtual high-speed link is split intofragments, one for each legacy access link. Overhead bits are added toeach fragment and sent together as a packet on each legacy data link.The overhead bits are used to properly reassemble fragments into datapackets of the virtual high-speed link, since they may arrive out oforder, due to the variable latency possible on each link. Multi-link PPPis such a protocol that creates a virtual high-speed link out of legacy(PPP) data link channel. In both of these bonding techniques, an entityin the central office or elsewhere is needed to bifurcate traffic in thedownstream direction and to reassemble traffic in upstream direction. Inthe case of physical layer bonding, special DSLAM line cards capable ofterminating bonded DSL lines are necessary. In the case of link layerbonding, for example multi-link PPP, a separate line card or server withmulti-link PPP support is needed to act as an end-point of the virtualhigh-speed link.

In intranet communications applications, a client device such as alaptop computer, desktop computer, or mobile phone, may have severalcommunication interfaces. For example it may have a wired Ethernetinterface and one or more wireless LAN interfaces (e.g., WiFi or IEEEStd. 802.11). For intranet communications, it may be desired toaggregate the communications resources available to the client in orderto achieve faster rates of data transfer, for a given communication tasksuch as a file transfer. The commonly known method for aggregation wouldbe to bond the interfaces together at the physical or the link layer. Ineither case, an entity is needed at both the client and at the remoteend of the virtual high-speed link, in order to bifurcate traffic on thedifferent communication channels at one end of the virtual high-speedlink, and reassemble traffic from the different communication channelsat the other end of the virtual high-speed link, for each direction oftraffic flow.

Many examples of apparatus and methods exist in the prior art toincrease data transmission speeds. For example, U.S. Pat. No. 6,490,295to Cooklev, et al. issued Dec. 3, 2002 and entitled “High-speed MODEMoperating over two or more telephone lines” discloses a communicationsystem configured to transceive a signal along multiple communicationmedia of the communication system thereby increasing the rate at whichthe signal is transceived. The communication system comprises a sourceconfigured to transceive a signal. A communication apparatus configuredfor decomposing the signal into a plurality of manipulated signals isalso disclosed. In communication with the communication apparatus is areconstructing apparatus that is configured for reconstructing theplurality of manipulated signals into the signal, the signal beingcapable of being transceived by a host.

U.S. Pat. No. 6,910,149 to Perloff, et al. issued Jun. 21, 2005 andentitled “Multi-device link aggregation” discloses a multi-device linkaggregation (MDLA). A first MDLA device is connected to a second MDLAdevice by an MDLA internal link. Each of the interconnected first andsecond MDLA devices is connected to at least one common link aggregation(LAG) partner device by an aggregated link, respectively. Typically, thefirst and second MDLA devices exchange protocol data units (PDUs) todetect devices that are connected to both the first and second MDLAdevices such that they are able to trick the detected devices intobehaving as though the two MDLA devices are a single device. If one ofthe MDLA devices fails, traffic of the associated aggregated link to thefailed MDLA device can be automatically forwarded by the othernon-failed MDLA device.

U.S. Pat. No. 7,002,993 to Mohaban, et al. issued Feb. 21, 2006 andentitled “Method and apparatus providing media aggregation in apacket-switched network” discloses techniques for aggregating multiplemedia packets to improve end-to-end bandwidth efficiency. The techniquesinclude using an RTP aggregation protocol that is not sensitive topacket loss to aggregate multiple media packets under a single header.According to the RTP aggregation protocol, the single header for anaggregated media packet comprises a version field, a zero field, asequence number field and a trunk ID field. The single headerencapsulates the aggregated payload, which is an aggregation ofReal-Time Protocol (RTP) segments. An RTP segment either has acompressed format or an uncompressed format. The uncompressed RTPsegment includes the complete uncompressed RTP packet copied from theoriginal User Datagram Protocol (UDP) packet. The compressed RTP segmentincludes the payload of the original RTP rather than the completeoriginal RTP packet.

U.S. Pat. No. 7,292,575 to Lemieux, et al. issued Nov. 6, 2007 andentitled “Method and system for multi-protocol label switching (MPLS)based data flow aggregation in a third generation (3G) cellulartelecommunication system” discloses a method and packet switchedcellular telecommunication system wherein data flows from a terminalinto one or more Multi Protocol Label Switching (MPLS) Label SwitchedPath (LSP) based on at least one criterion. In a first embodiment, thedata flows are aggregated into LSP(s) based on a Quality of Service(QoS) class of each such data flows, by a Network Node, such as a Radionetwork Controller (RNC) or a Serving GPRS Support Node (SGSN). In asecond embodiment, the data flows or LSPs are aggregated into anotherLSP(s) based on the destination routing area. The LSP aggregation isperformed by an Edge Node of the routing area, such as a Gateway GPRSSupport Node (GGSN). In a first variant of the 2.sup.nd embodiment, theaggregation is used for macro-mobility, while in yet another variant thesame aggregation is used for defining a virtual private network.

U.S. Pat. No. 7,315,554 to Baum, et al. issued Jan. 1, 2008 and entitled“Simple peering in a transport network employing novel edge devices”describes providing simple peering in a network in which transporttechnology is independent of network access technology. An out-of-bandnetwork may be used to carry advertisements to an update facility. Theupdate facility may use layer 3 destination address information and atleast a part of context information (to identify customers uniquely) todetermine a layer 3 address of an edge device of the transport networkassociated with an addressed customer device. Virtual private networksare supported, as context information is used to distinguish differentcustomers with overlapping layer 3 addresses.

United States Patent Publication No. 20060064492 to Hirsch; filed Mar.23, 2006 and entitled “Systems and methods for smart communication”discloses systems and methods for smart communication including a serverthat facilitates smart communication between users. Users involved in acommunication session using a first set of devices may, for example,automatically establish a second communication session between theusers, where the second communication session may be conducted using oneor more other devices associated with the users.

United States Patent Publication No. 20060084469 to Malone; et al. filedApr. 20, 2006 and entitled “Transmitter and receiver architecture formulti-mode wireless device” discloses a multi-mode wireless devicehaving a transceiver that includes a multi-mode transmitter and amulti-mode receiver. The transmitter can include a multi-mode transmitbaseband portion configured to support all of the transmit modes.Similarly, the receiver can include a multi-mode baseband portion thatis configured to support all of the receive modes. The transmitter canalso include a frequency conversion stage that can convert the outputfrom the transmit baseband portion to the desired transmit frequency.Multiple power amplifiers in parallel, each configured to support one ormore of the operating modes, can selectively amplify the transmitsignals. The receiver can include multiple low noise amplifiers (LNAs)in parallel, each configured to selectively amplify the received signalsof one or more of the operating modes. The output of the LNAs can becoupled to a frequency conversion stage that downconverts the receivedsignals and provides them to the baseband portion.

United States Patent Publication No. 20060215593 to Wang; et al. filedSep. 28, 2006 and entitled “Distributed Multichannel WirelessCommunication” discloses systems and methods that facilitate distributedmultichannel wireless communication and provide the highest levelquality of service (“QoS”) guarantee and support extremely highbandwidth applications such as voice over internet protocol (“VOIP”)streaming audio and video content (including high definition) as well asmulticast applications while also supporting convergent networks, ad hocnetworks, and the like. A modular MAC architecture provides a group ofnodes with the ability to simultaneously communicate with each otherusing multiple separate communication channels during the sametimeslots. The additional throughput gained by employing multiplecommunication channels is amplified by dynamically mapping thecommunication channels and timeslots in a network so that multiplechannels can be reused simultaneously throughout the network during thesame timeslot in a fashion that does not create collisions.

Despite the foregoing, improved methods and apparatus for aggregating aplurality of communication channels in a communication network isneeded. Ideally, such methods and apparatus would provide for aggregatedtransmission in a transparent manner. That is, such apparatus andmethods could be implemented in communication with an originating deviceand a destination device with neither of the devices being aware thatcommunication is provided over two or more separate communicationchannels. Ideally such improved methods and apparatus could beimplemented without requiring modifications to either the destinationdevice or the originating device and would provide a significant benefitin terms of inter alia not having to modify or retrofit the network orserver in any way.

SUMMARY OF THE INVENTION

In a first aspect of the present invention a method of aggregating aplurality of communication channels in a communication network isdisclosed. In one embodiment, the method comprises classifying aplurality of packets that arrive on a first communication interface asbelonging to a single communication session; selecting one of aplurality of network interfaces that will be associated with the singlecommunication session; forwarding a first packet received on the firstcommunication interface to the selected network interface associatedwith the single communication session that the first packet belongs to;updating a source network address of the first packet with the networkaddress associated with the selected network interface; and forwarding asecond packet received on the selected network interface fortransmission onto the first communication interface, wherein thedestination network address of the second packet is replaced by thesource network address associated with the single communication sessionthat the second packet belongs to. In another embodiment, the act ofselecting is based on varying levels of a packet security parameter.

In another embodiment, the method comprises requesting a plurality ofdata via a first communication interface, where the first communicationinterface is in communication with a first and a second networkinterface. The plurality of data comprises a first data object and asecond data object. The method further comprises receiving the firstdata object at the first network interface; receiving the second dataobject at the second network interface; and forwarding the first andsecond data objects to the first communication interface. In oneembodiment, selection of the first and second network interfaces isperformed using a load balancing algorithm. In yet another embodiment,the load balancing algorithm comprises a round robin load balancingalgorithm. In still another embodiment, the load balancing algorithmcomprises an optimization algorithm adapted to optimize for higherbandwidth channels. In still another embodiment, the load balancingalgorithm accounts for a quality of service (QoS) parameter.

In yet another embodiment, the method comprises classifying a firstplurality of packets that arrive on a first communication interface asbelonging to a single communication session, wherein the singlecommunication session represents a data transfer in the upstreamdirection. The method further comprises dividing a first packet receivedon the first communication interface belonging to the singlecommunication session into a plurality of fragments; and forwarding afirst fragment of the plurality of fragments over a first networkinterface and forwarding a second fragment of the plurality of fragmentsover a second network interface.

In a second aspect of the invention, apparatus for aggregatingcommunication channels in a packet based communication network accordingto the various methods of the first aspect of the invention isdisclosed. In one embodiment, the apparatus is embedded within a clientdevice. In another embodiment, the apparatus comprises a stand alonehardware device. In yet another embodiment, the apparatus is embeddedwithin a modem. In yet another embodiment, the apparatus is embeddedwithin a router. In still another embodiment, the apparatus is embeddedwithin a mobile device (e.g., PDA or cellular telephone). In stillanother embodiment, the apparatus utilizes a power consumption parameterto select one of a plurality of network interfaces.

In one variant, the aggregator apparatus comprises a computer (e.g., PC,laptop, handheld, etc.). In another variant, the aggregator comprises amobile device (e.g., cellular telephone). In still another variant, thedevice comprises a set-top box (STB) for use in, e.g., a satellitenetwork.

In yet another embodiment, the apparatus comprises a process, whereinthe process comprises: receiving a request for a plurality of data via afirst communication interface from a computing device, the plurality ofdata comprising a first data object and a second data object andbelonging to a single communication session; forwarding the first dataobject request to a first network interface; forwarding the second dataobject request to a second network interface; receiving the first dataobject at the first network interface; receiving the second data objectat the second network interface; and forwarding the first and seconddata objects to the first communication interface.

In one variant, the computing device is unaware of the acts of receivingthe first and second data objects at the first and second networkinterfaces, respectively. In yet another variant, the computing deviceis unaware of the acts of forwarding the first and second data objectrequests to the first and second network interfaces, respectively.

In yet another embodiment, the aggregator device comprises a portabledevice. The portable device may optionally be battery powered oralternatively powered by a wired or wireless power source. In oneembodiment, the communication channels in the portable device will beprovided by different mobile wireless broadband data services.Advantageously, the communication channels 41, 42 may be obtained fromdifferent communication carriers that will provide a communicationservice through the local port to an end user device which is higherperforming and has better geographical coverage than would be otherwiseavailable through any single communications carrier. The local port inmay comprise a wireless interface, or alternatively may comprise a wiredinterface.

In a third aspect of the invention, a system for aggregatingcommunication channels in a packet based communication network isdisclosed. In one embodiment, the packet based communication networkcomprises the Internet. In another embodiment, the packet basedcommunication network comprises an intranet.

In a fourth aspect of the invention, a proxy architecture is disclosed.In one embodiment, the architecture comprises an aggregator acting as aproxy for the data transfer request, transforming data transfer requestsfor comparatively large objects into a series of smaller data (chunk)transfer requests that are made through two or more network interfaces,and passing data chunks received over the different network interfaces,through to the original requestor (e.g., on a local port) in aprescribed order.

In a fifth aspect of the invention, a computer-readable medium isdisclosed. In one embodiment, the medium comprises a storage deviceadapted to store at least one computer program comprising an aggregationbalancing or allocation algorithm adapted for use in a network. In onevariant, the medium comprises a computer hard disk (HDD). In anothervariant, the medium comprises an optical device. In yet anotherembodiment, the medium comprises a USB key. In still another embodiment,the medium comprises a computer (e.g., flash) memory.

In a sixth aspect of the invention, a method of avoiding the enactmentof a false congestion avoidance mode of a transmitting source device isdisclosed. In one embodiment, the method comprises forwarding aplurality of data packets from a local port over a plurality of networkinterface ports; receiving an acknowledgment message comprising asequence number at one of the plurality of network interface ports;determining whether the sequence number is identical to a previouslyreceived acknowledgment message; and selectively discarding theacknowledgement message so as to avoid the transmitting source devicefrom unnecessarily entering a congestion avoidance mode.

In another embodiment, the act of selectively discarding theacknowledgment message is performed utilizing a timer mechanism.

In yet another embodiment, the act of forwarding comprises: classifyingthe plurality of data packets that arrive on the local port as belongingto a single communication session; selecting one of the plurality ofnetwork interfaces that will be associated with the single communicationsession; forwarding a first packet associated with the singlecommunication session to a first of the plurality of network interfaceports; updating a source network address of the first packet with thenetwork address associated with the first network interface port; andforwarding a second packet associated with the single communicationsession to a second of the plurality of network interface ports, whereinthe source network address of the second packet is replaced by thesource network address associated with the first network interface port.

In yet another embodiment, the method further comprises analyzing thetiming of the receipt of the acknowledgement message and the previouslyreceived acknowledgment message so as to determine whether: (1) theacknowledgment messages are related; or (2) the acknowledgment messagesindicate a congestion condition.

In a seventh aspect of the invention, methods of doing business aredisclosed which implement various aspects of the above-mentioned methodsand apparatus.

In an eighth aspect of the invention, network apparatus for use in apacketized data network and, adapted to selectively utilize two or moreavailable and substantially heterogeneous communication channels forcommunicating data associated with a single communications session so asto enhance the cost efficiency of said session is disclosed.

In a ninth aspect of the invention, a computer-readable apparatus isdisclosed. In one embodiment, the apparatus comprises a storage mediumadapted to store a computer program comprising a plurality ofinstructions which, when executed on a digital computer, provide foraggregation of a plurality of different packetized network interfaces inorder to enhance cost efficiency for a user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of a device implementing methods in accordancewith the invention, whereby device aggregates multiple communicationchannels in order to communicate with other devices through a packetswitched network.

FIG. 1a is an illustration of a device implementing methods inaccordance with the invention, whereby device aggregates multiplecommunication channels over various packet switched and packet overcircuit networks.

FIG. 2 is a logical flow diagram illustrating a first exemplary methodof overwriting IP address fields in accordance with the principles ofthe present invention.

FIG. 3 is a logical flow diagram illustrating a first exemplary methodof determining whether to use multiple network interfaces for datacommunication in accordance with the principles of the presentinvention.

FIG. 4 is a logical flow diagram illustrating a first exemplary methodof overwriting a source network IP address field with the same valueregardless of the network interface utilized in accordance with theprinciples of the present invention.

All figures © Copyright 2007-2008 Mushroom Networks, Inc. All rightsreserved.

DETAILED DESCRIPTION

Reference is now made to the drawings wherein like numerals refer tolike parts throughout.

As used herein, the terms “client device” and “end user device” include,but are not limited to, set-top boxes (e.g., DSTBs), personal computers(PCs), and minicomputers, whether desktop, laptop, or otherwise, andmobile devices such as handheld computers, PDAs, personal media devices(PMDs), and smartphones.

As used herein, the term “computer program” or “software” is meant toinclude without limitation any sequence or human or machine cognizablesteps which perform a function. Such program may be rendered invirtually any programming language or environment including, forexample, C/C++, Fortran, COBOL, PASCAL, assembly language, markuplanguages (e.g., HTML, SGML, XML, VoXML), and the like, as well asobject-oriented environments such as the Common Object Request BrokerArchitecture (CORBA), Java™ (including J2ME, Java Beans, etc.), BinaryRuntime Environment (BREW), and the like.

As used herein, the term “digital subscriber line” (or “DSL”) shall meanany form of DSL configuration or service, whether symmetric orotherwise, including without limitation so-called “G.lite” ADSL (e.g.,compliant with ITU G.992.2), RADSL: (rate adaptive DSL), VDSL (very highbit rate DSL), SDSL (symmetric DSL), SHDSL or super-high bit-rate DSL,also known as G.shdsl (e.g., compliant with ITU Recommendation G.991.2,approved by the ITU-T February 2001), HDSL: (high data rate DSL), HDSL2:(2nd generation HDSL), and IDSL (integrated services digital networkDSL), as well as In-Premises Phoneline Networks (e.g., HPN).

As used herein, the term “Ethernet” refers generally to local areanetwork technology; including that now specified in a standard IEEE802.3 and related standards.

As used herein, the term “integrated circuit (IC)” refers to withoutlimitation any type of device, whether single or multiple die, havingany level of integration (including without limitation ULSI, VLSI, andLSI) and irrespective of process or base materials (including, withoutlimitation Si, SiGe, CMOS and GaAs). ICs may include, for example,memory devices (e.g., DRAM, SRAM, DDRAM, EEPROM/Flash, ROM), digitalprocessors, SoC devices, FPGAs, ASICs, ADCs, DACs, transceivers, memorycontrollers, and other devices, as well as any combinations thereof.

As used herein, the terms “Internet” and “internet” are usedinterchangeably to refer to inter-networks including, withoutlimitation, the Internet.

As used herein, the term “Internet Protocol” or “IP” refers generally tothe format of packets, also called datagrams, and/or an addressingscheme, such as without limitation those described in inter alia, RFC791 and 2460. IP may be used as an identifier for a computer or deviceon a TCP/IP or comparable network; e.g., such as networks using TCP/IProute messages based on the IP address of the destination.

As used herein, the term “memory” includes any type of integratedcircuit or other storage device adapted for storing digital dataincluding, without limitation, ROM. PROM, EEPROM, DRAM, SDRAM, DDR/2SDRAM, EDO/FPMS, RLDRAM, SRAM, “flash” memory (e.g., NAND/NOR), andPSRAM.

As used herein, the terms “microprocessor”, “processor” and “digitalprocessor” are meant generally to include all types of digitalprocessing devices including, without limitation, digital signalprocessors (DSPs), reduced instruction set computers (RISC),general-purpose (CISC) processors, microprocessors, gate arrays (e.g.,FPGAs), PLDs, reconfigurable compute fabrics (RCFs), array processors,secure microprocessors, and application-specific integrated circuits(ASICs). Such digital processors may be contained on a single unitary ICdie, or distributed across multiple components.

As used herein, the term “network” refers generally to any type oftelecommunications or data network including, without limitation, datanetworks (including MANs, WANs, LANs, WLANs, internets, and intranets),hybrid fiber coax (HFC) networks, satellite networks, and telconetworks. Such networks or portions thereof may utilize any one or moredifferent topologies (e.g., ring, bus, star, loop, etc.), transmissionmedia (e.g., wired/RF cable, RF wireless, millimeter wave, optical,etc.) and/or communications or networking protocols (e.g., SONET,DOCSIS, IEEE Std. 802.3, ATM, X.25, Frame Relay, 3GPP, 3GPP2, WAP, SIP,UDP, FTP, RTP/RTCP, H.323, etc.)

As used herein, the term “network entity” refers to any network entity(whether software, firmware, and/or hardware based) adapted to performone or more specific purposes. For example, a network entity maycomprise a computer program running in server belonging to a networkoperator, which is in communication with one or more processes on a CPEor other device.

As used herein, the term “network interface” refers to any signal, data,or software interface with a component, network or process including,without limitation, those of the Firewire (e.g., FW400, FW800, etc.),USB (e.g., USB2), Ethernet (e.g., 10/100, 10/100/1000 (GigabitEthernet), 10-Gig-E, etc.), MoCA, Serial ATA (e.g., SATA, e-SATA,SATAII), Ultra-ATA/DMA, Coaxsys (e.g., TVnet™), radio frequency tuner(e.g., in-band or OOB, cable modem, etc.), WiFi (802.11a,b,g,n), WiMAX(802.16), PAN (802.15), or IrDA families.

As used herein, the term “protocol” refers generally to a standardformat agreed upon for communication between entities, such asdistributed systems within a network. Examples of protocol include(Transmission Control Protocol (TCP), Internet Protocol (IP), TCP/IP,Hypertext Transmission Protocol (HTTP), Session Initiation Protocol(SIP), Real Time Transport Protocol, Real Time Control Protocol(RTP/RTCP), etc.)

As used herein, the term “server” refers to any computerized component,system or entity regardless of form which is adapted to provide data,files, applications, content, or other services to one or more otherdevices or entities on a computer network.

As used herein, the term “TCP/IP” refers generally to TransmissionControl Protocol/Internet Protocol, the suite of communication protocolsused to connect hosts on the Internet.

As used herein, the term “WiFi” refers to, without limitation, any ofthe variants of IEEE-Std. 802.11 or related standards including 802.11a/b/g/n.

As used herein, the term “wireless” means any wireless signal, data,communication, or other interface including without limitation Wi-Fi,Bluetooth, 3G, HSDPA/HSUPA, TDMA, CDMA (e.g., IS-95A, WCDMA, etc.),FHSS, DSSS, GSM, PAN/802.15, WiMAX (802.16), 802.20, narrowband/FDMA,OFDM, PCS/DCS, analog cellular, CDPD, satellite systems, millimeter waveor microwave systems, acoustic, and infrared (i.e., IrDA).

Overview

In one fundamental aspect, the present invention details methods andapparatus for aggregating a plurality of communication channels and/ornetworks to improve the speed and reliability of networked connectivitybetween computing devices. The present invention also seeks to providethese benefits while minimizing the drawbacks associated with theirimplementation by essentially making the implementation of theaforementioned methods and apparatus transparent to the pre-existingnetwork architecture. The present invention in effect intercepts datatraffic associated with a network and redistributes that traffic basedon any number of considerations, including without limitation, loadbalancing concerns, quality of service considerations, cost and thelike. Further the apparatus and methods of the present invention alsodetail associated algorithms and apparatus that manage pre-existingmechanisms associated with the network that attempt to addressconnectivity considerations such as congestion avoidance. Detailsassociated with the implementation of these methods and apparatus arenow described in detail.

Methods of Doing Business

An aggregation device in accordance with the principles of the presentinvention may provide for accelerated network connectivity in acost-effective way to small and medium size businesses, enterprises,apartments, hotels, hospitals and other Multi-tenant buildings who wishto aggregate multiple Internet access lines for increased performanceand reliability. Accordingly a method of doing business is envisioned.With an aggregation device, multiple DSL, cable modem, T1 services, etc.can be combined to provide accelerated and more reliable Internet access(via redundancy and the like). The Internet access speeds provided bythe aggregation device may, in certain geographical areas, not even beavailable from the service provider. It has been noted by the assigneeof the present invention that the total subscription costs of multipleInternet access lines are often substantially less than a singlehigh-speed Internet access line possessing equivalent bandwidth.Accordingly, the aggregation device, in many application scenarios,provides quick Return on Investment (ROI) and lower operating expensesfor a variety of customers. In some instances, the payback period on theinvestment of an aggregation device by a customer is often less than 6months.

Exemplary Embodiments of Apparatus

The invention is described relative to the exemplary configuration ofFIG. 1. An aggregation device 101 is connected to a network (e.g., apacket switched network 1000) through a plurality of communication links41, 42. Further, while two (2) communication links 41, 42 are shown inFIG. 1, it is recognized that three (3) or more communication linkscould be utilized if desired. Moreover, these links need not be separatephysical links, but may in fact comprise virtual links or channels ofthe type know in the digital communications arts. Additionally, it willbe appreciated that the links 41, 42 need not be direct links, but infact may comprise indirect links (e.g., may be routed through one ormore intermediary devices or processes, or even other networks), and maybe either wireline or wireless in nature.

Other devices, represented in the illustration as servers 201, 202, 203,are connected to the packet switched network 1000 through theirrespective communication links 51, 52, 53. The aggregation device 101 isadapted to provide network communications to devices that are connectedto the aggregation device 101 via a communications port 11, to otherdevices such as those devices 201, 202, 203 connected to the packetswitched network 1000. Throughout this disclosure and for purposes ofillustration, we assume that the destination device is the serverlabeled as 201, however it is recognized that the destination device maycomprise literally any number of devices whether alone or in combinationthat are connected directly or indirectly to the packet switched network1000.

The communication links 41 and 42 are aggregated together by theaggregation device 101 to communicate with other devices connected tothe packet switched network, and hence the name aggregation device. Byaggregating the communication links together, greater data transferrates can be achieved in both the upstream and downstream direction,versus communication over a single communication link. Thisfunctionality is described in greater detail subsequently herein.

In the embodiment illustrated in FIG. 1, there are only twocommunication links 41, 42 that the aggregation device 101 can use forcommunication with other devices connected to the packet switchednetwork 1000. Utilizing a conventional approach to aggregate thecommunication channels, it might be desirable to employ an aggregatingtype device within the packet switched network 1000 itself, and anotheraggregating type device within a destination device, such as device 201.Therefore, for a given data transfer, a first aggregation type devicewill bifurcate the data traffic so that it is spread across a pluralityof communication channels, and a second aggregation type device wouldreassemble the bifurcated traffic back into a single data stream. Thetwo aggregation type devices would thus work together to form a virtualhigh-speed communications link between them. Utilizing this approach,the device connected to the local port 11, and the device that it iscommunicating with, for example the device 201, is not aware thatcommunication is being provided over two or more separate communicationchannels. As a result, no changes to the device connected to local port11 are required to support the communication over two separatecommunication channels. In the case where the similar aggregation deviceis deployed within the packet switched communication network 1000, nochange is required in the destination device 201 in order to support thecommunication over two separate communication channels. In the casewhere the similar aggregation device is deployed within the destinationdevice 201, no change is required within the destination softwareapplication within the destination device 201.

Contrast the foregoing approach with another exemplary method of thepresent invention, where advantageously there is no such similaraggregation type device that is necessary to deploy in the packetswitched network 1000 or in the destination device 201. Rather, theaggregation device 101 of FIG. 1 is solely responsible for aggregatingthe communication channels 41, 42. The aggregation device 101 providesfor transparent data transport, i.e. the device connected to the localport 11, and the destination device 201, are not aware thatcommunication is provided over two separate communication channels 41,42. In the context of the present invention, the term local port 11refers to the fact that the device transmitting or receiving data willtend to be more physically proximate to the aggregation device 101 thenthe destination device, although it is recognized in certain embodimentsthat this is not necessarily a requirement. Regardless, there are nomodifications required in either the destination device 201 or in thedevice connected to the local port 11 in order to support thecommunication using the separate communication channels 41, 42. Thisprovides a significant benefit in terms of inter alia not having tomodify or retrofit the network 1000 or server 201 in any way,eliminating many barriers to the implementation of the aggregationdevice 101.

Referring now to FIG. 1a , an alternative configuration of the inventionis shown and described in detail. In the embodiment illustrated in FIG.1a , the aggregation device is coupled via two communication channels42, 41 to a first and a second packet switched network 2000 and 1000respectively. The second packet switched network 1000 is coupled to athird packet switched network 3000 which is coupled to a plurality ofdestination devices 201, 202, 203 via respective communication links 51,52 and 53. The third packet switched network 3000 is coupled to thefirst packet switched network 2000 via a gateway 60. The first packetswitched network 2000 is also coupled to a packet over circuit network4000 via a second gateway 70. Coupled to the packet over circuit network4000 is a packet-enabled circuit switched device 80 which can nowcommunicate with the aggregation device 101 which can alsosimultaneously communicate with one or more of the destination devices201, 202, 203. In this manner, the device 101 of the present inventionmay communicate with any number of devices via a plurality of similarand differing network architectures.

Example Application Scenarios

The present invention may be utilized in any number of exemplarynetworking applications. In a first example application, called hereinthe “ISP application,” the packet switched network 1000 represents theInternet. The aggregation device 101 will comprise a separate hardware,firmware and/or software device deployed by an individual or a businessin order to aggregate two or more Internet access lines. For example,the aggregation device 101 may comprise in one variant the Truffle™Broadband Bonding Network Appliance (BBNA) or the APX10 Access PointAggregator designed and manufactured by the Assignee of the presentdisclosure. The local port 11 may comprise for example a wired EthernetLAN port or other network interface, and the communication channels 41,42 comprise wired Ethernet connections or other interfaces tomodulator/de-modulator devices such as two or more DSL modems that areconnected to the Internet. The destination devices 201, 202, 203 mayrespectively comprise computing devices ubiquitous in the computing artssuch as file servers, web servers, remote personal computers, etc, whichthe individual or business wishes to communicate with.

In another exemplary embodiment the aggregation device 101 may beembedded within an Internet router deployed by a business that wishes toemploy two or more broadband lines (e.g. DSL) for Internet connectivity.In yet another exemplary embodiment, the aggregation device 101 may beintegrated with two or more broadband access devices (such as legacy DSLmodems, cable modems, etc.), with the aggregation device 101 itselfoperating with the same functionality as a broadband modem. Thecommunication channels 41, 42 may represent, inter alia, DSL lines (i.e.twisted copper pairs running from an individual or business to a centraloffice or “CO” or to a remote terminal of the communication carrierproviding the DSL service).

In yet another exemplary embodiment, hereinafter referred to as the“home intranet application,” the packet switched network 1000 comprisesa data network implemented inside of a consumer's residence. Theaggregation device 101 may comprise a separate hardware device (such asa PMCIA card, USB key, etc.) that plugs into, for example, a laptopcomputer or a desktop computer. In this case, the local port 11represents a PMCIA bus connector and the aggregation device 101 acts asa network interface card that aggregates two separate networkconnections. The network connections 41, 42 might be wireless LANconnections, or wired Ethernet connections, or a combination of both. Inthe home intranet application, the destination devices 201, 202, 203might represent any number of devices utilized in a home intranetapplication such as a desktop or laptop computer, a television set-topbox, a video game console, personal media device (PMD), etc.

In yet another embodiment in e.g. a home intranet application, theaggregation device 101 may be embedded within a laptop or handheldcomputer or a desktop, known as a client. In this case, the local port11 may be an internal software port of the client. The communicationlinks 41, 42 might represent wireless LAN connections or wired LANconnections that are emanating from the client as previously discussed.In this case, as would be appreciated by one of ordinary skill, themethods of the invention discussed herein may be implemented entirely insoftware within the client. This client may be resident on the hostdevice, or alternatively on another device in data communication withthe host (via e.g., a high speed serialized interface such as USB orFirewire). It may also be downloaded and installed to the host (such asby a network operator), or disposed on the host in any number of otherways (e.g., CD-ROM, at time of manufacture, etc.).

In yet another exemplary embodiment, the aggregation device 101 may be aportable battery powered device (such as the exemplary “Portabella BBNA”manufactured by the Assignee hereof). In one embodiment, thecommunication channels 41, 42 in the portable battery powered devicewill be provided by different mobile wireless broadband data services.Advantageously, the communication channels 41, 42 may be obtained fromdifferent communication carriers (e.g. Sprint, Verizon, ATT, etc.). Inthis case the aggregation device 101 will provide a communicationservice through the local port 11 to an end user device which is higherperforming and has better geographical coverage than would be otherwiseavailable through any single communications carrier. The local port 11in this embodiment may comprise a wireless interface such as a WiFiinterface, or alternatively may comprise a wired interface such as anEthernet interface. The portable device embodiments discussed previouslyabove have a wide number of varying applications, including emergencyresponse, law enforcement, limousines, kiosks, construction sitecommunications, etc. More generally, the above mentioned portable deviceembodiment is useful anywhere a fast and reliable portable networkconnection (i.e. wireless) is needed.

Components of Aggregation Device

For purposes of clarity in describing the present invention, theexemplary embodiment of the aggregation device 101 can be thought of ascomprising a local port 11, a processor 100, a link 21 to a firstnetwork interface 31, a second link 22 to a second network interface 32,a communication link 41 for the first network interface 31, and a secondcommunication link 42 for the second network interface 32. Both externalinterfaces in this embodiment have an associated network level addressand a link (MAC) layer address, although other network addressingschemes may be used as well. The device may also include any number ofother components (not shown) including tuners and demodulator/decryptorsfor interface with a cable RF (e.g., HFC) network, digital processor(s),storage device/memory, and a plurality of interfaces (e.g., video/audiointerfaces, IEEE-1394 “Firewire”, USB, serial/parallel ports, etc.) forinterface with other end-user apparatus such as televisions, personalelectronics, computers, WiFi or other network hubs/routers, etc. Othercomponents which may be utilized within the device (deleted from FIG. 1for simplicity) are various software processing layers as well as mediaprocessors and other specialized SoC or ASIC devices. These additionalcomponents and functionality are well known to those of ordinary skillin the cable and embedded system fields, and accordingly not describedfurther herein.

The aggregator 101 may also be provided with e.g., Java-based middlewarewhich, inter alia, manages the operation of the device and applicationsrunning thereon (including the client process previously described). Itwill be recognized by those of ordinary skill that myriad differentdevice and software architectures may be used consistent with the tuningfunctions of the present invention, the device of FIG. 1 being merelyexemplary.

System Operation: Downstream Data Transfers

We first describe one embodiment of how the aggregation device 101 worksto aggregate the communication links 41, 42 for data transfers in thedownstream direction, i.e. data transfers originating from one or moredestination devices 201. A key observation is that data transfers in thedownstream direction are almost always “pull”-based, rather than“push”-based. In other words, the recipient requests the data transferfrom the source, rather than the source transferring the data to therecipient in an unsolicited manner. It will be appreciated, however,that while described primarily in the context of a “pull”, the inventionis in no way so limited, and may also be implemented within “push”paradigms, such as e.g., an exemplary WAP push of the type well known inthe wireless arts.

Many applications of data downloading generate bursts of data downloadsfor data objects which are typically downloaded in parallel. Forexample, in an exemplary web browsing application, a user may request toview a web page. The web page might have several objects (e.g.multimedia objects, text, etc.), and the user's web browser translatesthe page request into several download requests, one for each object.The individual objects of the web page may be located on differentremote servers, and often all of the download requests for theindividual objects of the web page may be issued before the requestedobjects arrive at the web browser. As a result, the objects areeffectively downloaded in parallel. For such applications, theaggregation device may simply forward data download requests received onthe local port 11 through one of the communication links 41 or 42. Inthis case, the source device of the object download request, say device201, believes the object was requested by one of the network interfaces,31 or 32, since the value of the source address in the object downloadrequest is either the network address of interface 31 or the networkaddress of interface 32. The aggregation device 101 can forward asequence of data download requests for different objects according to analgorithm that attempts to, inter alia, evenly distribute load acrossthe communication links 41, 42. In one embodiment, the aggregationdevice 101 may forward object download requests through thecommunication links in a round-robin type manner. In another embodiment,if one of the communication links 41, 42 has a faster data rate than theother, it may be desirable to forward object download requests at ahigher rate through the faster communication link.

In another embodiment, the aggregation device 101 may forward objectdownload requests of one type exclusively through one of thecommunication links. This could be useful for example, in objecttransactions that require differing levels of security. Onecommunication link may be equipped with a firewall, encryptioncapability, a physically secure (e.g., protected) link to the network,VPN capability, IEEE-Std. 802.1x/WPA2, WTLS, or other means of providingadditional layers of security to the object transaction and hence, dataobjects of that type may be funneled exclusively through that securecommunication link.

Similarly, different links or interfaces might impose different QoS(quality of service) or forward error correction (FEC) algorithms (suchas Turbo coding, Reed-Solomon, or Viterbi), and hence may be selectivelyallocated on this basis as well.

Similarly, the device 101 may also selectively route certain types ofdata (e.g., packets or other data structures according to a prescribedprotocol) through one of the links as opposed to another. The selectedroute may be chosen based on data requirements for throughput, latency,total download speed, robustness of linkage, lossiness of datatransmission, security and/or requirements for different physical layermethods (e.g. QPSK, 16 QAM, etc.). In one embodiment, traditionallyhigh-bandwidth data can be passed over one interface and lower bandwidthdata over the other, such as in multimedia or gaming applications wherethe indigenous processing requirements for the different types of dataare heterogeneous. Coded or encrypted data may be segregated in thisfashion as well; e.g., MPEG2 on one interface, MPEG4 on anotherinterface, etc.). Myriad variations on this basic theme of selectivelyallocating data to (or receiving from) one or more of a plurality ofinterfaces will be recognized by those of ordinary skill provided thepresent disclosure.

In another embodiment, the device 101 may selectively allocate certaintypes of data through one of the links as opposed to another in order toperform an additional processing function within the aggregator device101 itself that may not necessarily be available if allocated to anotherof the links.

In the case where the packet switched network represents the Internetand utilizes a protocol such as the ubiquitous IP protocol, the networkaddresses of the various entities are known as IP addresses. Dataobjects may be transported using, for example, the TCP protocol. A TCPsession may be opened for each data object transfer, and the aggregationdevice 101 may detect SYN packets in order to recognize the datadownload requests. A SYN packet arriving on the local port 11 may simplybe forwarded through a link 21 or 22 to the chosen network interface. Inthis case the processor 100 within the aggregation device 101 keeps arecord of which network interface a particular SYN packet was forwardedon, so that all traffic arriving on the local port 11 that is associatedwith SYN packet and the object download request is forwarded through thesame network interface. Such traffic associated with the same downloadrequest is commonly called a “flow,” and data packets belonging to thesame flow have the same source IP address, source port, destination IPaddress, destination port, and protocol ID. This can be used to identifypackets belonging to the same flow. When a data packet from a given flowarrives on the local port 11, the value of the source IP address fieldis overwritten by the aggregation device with the value of the IPaddress belonging to the selected network interface where the SYN packetwas sent though. Conversely, when a data packet from a given flowarrives on a network interface, the value of the destination IP addressis overwritten with the value of the source IP address used in theoriginal SYN packet that started the flow. Using this mechanism, as faras the remote device 201 is concerned, the requestor of an object is thechosen network interface of the aggregation device 101. Similarly, thedevice attached to the local port 11 is not aware of the redirectiondownload requests by the aggregation device 101 to a chosen networkinterface, 31 or 32.

An exemplary method for illustrating such a process flow 200 isillustrated at FIG. 2. The process flow 200 of FIG. 2 starts withreceiving data over a local port 11 at step 204. At step 205 theprocessor 100 of the device 101 utilizes an algorithm to choose anetwork interface 31, 32 for use with the data transfer. At step 206,the received data comprises a source IP address associated with theoriginating device located over the local port 11 is overwritten with anew source IP address associated with the chosen network interface 31,32. The data is then transmitted over the chosen network interface. Datareceived in response to this data being sent is then received over thenetwork at the chosen network interface at step 208. The processor 100implements an algorithm to overwrite the destination IP address of thereceived data with the address of the originative device coupled to thelocal port 11. In this manner, data sent and received via theaggregation device 101 is transparent to the originating device.

In some embodiments, the packet format utilized is handled by theaggregator based on network layer format requirements chosen per datastream. In one embodiment, the network layer format requirements may bepre-stored in memory on the aggregation device 101. In anotherembodiment, the network layer format requirements may be queried fromthe network 1000 by the aggregation device 101. In yet anotherembodiment, the network layer format requirements may be queried fromthe device connected over the local port 11 by the aggregation device101. In yet another embodiment, the network layer format requirementsmay be queried from the destination device 201 by the aggregation device101.

The aggregation device 101 can also obtain information regardingattributes of the requested data object by interpreting semantics withinthe file transfer protocol. In particular, by examination of the datapackets arriving on the local port 11, the aggregation device 101 candetermine the size of the requested object in a download request. Fordata objects smaller than a threshold, all of the traffic associatedwith the data transfer can be redirected by the processor 100 to asingle network interface, according to the method described above. Fordata objects larger than the threshold, the aggregation device 101 maychoose to use both network interfaces 31, 32 in order to download thefile, if possible.

An exemplary method 300 of implementing the aforementioned thresholdalgorithm is illustrated at FIG. 3. At step 302, attributes associatedwith the requested data object are determined. At step 304, it isdetermined whether or not the requested data object attributes exceed apredetermined threshold. If the attributes exceed the threshold,multiple network interfaces are utilized for the data object request atstep 306. If not, a single network interface is utilized at step 308.

For such large data objects, the processor 100 handles a downloadrequest arriving on the local port 11 as follows. Many file serverssupport a feature called “ranging,” and the aggregation device 101 candetermine if the object of a download request is “range-able.” If so,the aggregator device 101 uses both network interfaces 31, 32 totransfer the data object as follows. The processor 100 issues newdownload requests for different segments, or different “ranges,” of thedata object, called “chunks.” Thus, different parts of the requesteddata object are transferred through separate network interfaces, and itappears to the remote device 201 that different parts of the data objectare being requested by different network interfaces 31, 32. Regardlessof the order in which requested data arrive to the aggregator device 101from the packet switched network 1000, the requested data must bepresented to the original source of the download request on the localport 11 in the same order it would be presented had it been transferredover a single network interface. Thus, the processor 100 uses a memoryor storage device (such as a FIFO, queue or other buffer) to storerequested data chunks arriving out of order from the packet switchednetwork 1000. The aggregation device 101 can balance the load over thenetwork interfaces 41, 42 in a dynamic manner, depending on theinstantaneous data transfer speed observed on each network interface.For example, if an outstanding requested chunk takes longer to downloadthan a given threshold, the requested transfer of the chunk can beaborted and the same chunk can be re-requested though a differentnetwork interface. Alternatively, if the requested data is considered“lossy” (such as e.g., where QoS considerations are less critical, as incertain VoIP or other applications where the loss of some data is notfatal), the chunk can be merely discarded. If the speeds of the networkinterfaces are different, in order to balance load and minimize totaldownload time for the object it is desirable to transfer a largerfraction of the data object through the faster network interface.

The assignment of chunks to network interfaces can be performeddynamically (such as via e.g., a software or other control/allocationprocess running on the device 101), in order to appropriately balancethe load on the communication links 41, 42. In one embodiment, datachunks that arrive to the aggregation device 101 from the packetswitched network 1000 out of order are temporarily stored within amemory buffer inside the processor 100, and ultimately delivered overthe local port 11 in the correct order (via, e.g., a packet or framesequence number, timing information, or other approaches well known inthe digital communications arts) at the earliest possible opportunity.In effect, the aggregation device 101 acts as a proxy for the datatransfer request, transforming data transfer requests for large objectsinto a series of smaller data (chunk) transfer requests that are madethrough the network interfaces 31, 32, and passing data chunks receivedover the different network interfaces 31, 32, through to the originalrequestor on the local port 11 in the correct order. Since differentchunks may be transferred across the network interfaces in parallel, thetotal transfer time for the requested data object may be much smallerthan it would be if only a single network interface was used for thedata transfer.

If the processor 100 determines that the size of a data object requestedfor download on the local port 11 is above the threshold (or meetsanother criterion used for evaluation), but that the data object is notrange-able, the processor can simply forward all of the trafficassociated with the data transfer through a single network interface, asin the case for sufficiently small data objects discussed above.

Downstream data transfers can also be redundantly assigned to two ornetwork interfaces 31, 33 simultaneously to improve data transfer speedsfor objects sensitive to QoS considerations. In this manner, theaggregator device 101 can discard packets received over both networkinterfaces which are redundant. However, if data packets are lost overone communication link 41 while received at the other 42, then theaggregator device 101 can ignore known signaling messages that areindicative of congestion which might adversely impact data downloadspeeds on the communication link 41 which lost the packet, etc. Ageneral discussion of the methods and apparatus for ignoring congestionsignaling messages are discussed subsequently herein below.

System Operation: Upstream Data Transfers

For data transfers in the upstream direction, e.g., where the initiatorof data transfer is the destination device 201, the exemplaryaggregation device 101 can choose a single network interface (such asI/F 31) to facilitate the data transfer in the upstream direction forpackets belonging to the same flow. Thereafter, data packets from agiven flow arriving on the local port 11 can be spread across thedifferent network interfaces 31, 32 according to any number ofload-balancing or allocation algorithms. For example, a first exemplaryload-balancing algorithm may wish to route a larger fraction of datapackets from a flow over the network interface which is capable offaster communication with the destination device 201, as previouslydiscussed.

Recall that with the exemplary embodiments of the invention, there is noneed for a similar aggregation device in the packet switched network1000 or remote device 201 that acts as a peer to the aggregation device101. In fact, as previously discussed, the remote device 201 is unawarethat the upstream data transfer is taking place in parallel overdifferent communication links 41, 42, and so it is necessary to arrangematters so that data packets arrive to the remote device 201 in anappropriate manner that will be indistinguishable from the manner inwhich they would arrive had the transfer taken place over a singlecommunication link, either 41 or 42. Two methods are disclosed forupstream data transfer, namely network layer reassembly and transportlayer reassembly. We describe the methods in terms of the Internetprotocol, and where the network layer corresponds with IP, and thetransport layer corresponds with TCP. It is recognized however that oneof ordinary skill could apply the presently disclosed principles to anynumber of different networking architectures.

Upstream Data Transfers with Network Layer Re-Assembly

The exemplary IP networking protocol (as well as other protocols) has afeature to support the fragmentation of IP packets within a network,with re-assembly at the destination device of fragments belonging to thesame original IP packet. The fragmentation feature was initiallymotivated by transport of packets through heterogeneous networks, whichin general may have different limitations as to the maximum size ofpackets that can be transported over the links which comprise thenetwork. An incoming IP packet to a network may be too large to becarried natively by the network, so the IP protocol specification allowsan IP packet to be fragmented, i.e. divided into smaller pieces,en-route to its destination. Special control bits within an IP packetindicate if the packet is a fragment of a larger packet, and are usedfor proper reassembly at the destination of all the fragments belongingto the original packet.

As a result of this mechanism, almost all legacy IP devices have theembedded capability of reassembling IP packet fragments that areultimately destined for the IP device. The present inventionadvantageously leverages this extant capability by using thisre-assembly feature at the destination device 201 for supportingparallel data transfers over different communication links. Inparticular, the aggregation device 101 first identifies IP packetsarriving on local port 11 which belong to the same flow for an upstreamdata transfer. For each such IP packet belonging to a given flow, the IPpacket is split into two fragment IP packets and each fragment isforwarded separately over a different network interface 31 or 32 to thedestination device 201. The value of the source IP address field in eachfragment IP packet in a given flow is the same, and is either theoriginal IP address of the source device, or the IP address of exactlyone of the network interfaces, 31 or 32. As a result of this bifurcationmechanism, the fragments comprising a given IP packet that arrived onthe local port 11 can be correctly reassembled at the ultimatedestination device 201. This can be accomplished, as previouslydescribed, with effectively no changes to the destination device 201.

FIG. 4 illustrates an exemplary method 400 for implementing such aprocess flow. At step 402 data for transfer in the upstream direction isreceived by the aggregation device 101 over the local data port 11. Thedevice 101 selects a network interface based on any number ofconsiderations including a load balancing algorithm at step 404. Thedata is then forwarded to a respective network interface at step 406 andthe value of the source network IP address is written with either the IPaddress of the originating device or the IP address associated withexactly one of the plurality of network interfaces so that the datareceived at the destination device looks the same regardless of thenetwork interface which actually sent the data at step 408.

In practice, fragments may only rarely arrive out of order, anddestination devices may be optimized (e.g., buffer allocations made,etc.) on the basis of this assumption. Therefore, it may be desirablethat the timing of outgoing fragments from the aggregation device 101 bearranged such that with high probability, the fragments arrive at theultimate destination 201 in the same order as they would have been withthe ordinary fragmentation mechanism within the packet switched network1000. However, this is by no means a requirement for practicing theinvention, especially in more “lossy” applications where a comparativelysmall number of out-of-order packets will not affect the overallcommunication quality or effectiveness significantly. Moreover, inanother variant of the invention, the optimizations made in thedestination device may be used in a dynamic fashion; e.g., where thesource device communicates to the destination device that it is usingaggregation (such as via a packet, signal, flag bit in a protocol field,etc.), and the destination device accordingly relaxing its optimizationso as to provide a more robust and tolerant link.

In the ISP application, the packet switched network 1000 represents theInternet, and the endpoints of the communication links 41, 42 terminatewithin a point of presence operated by an Internet Service Provider(ISP). In this case, it may be important that the ISP does not “filter”(i.e. selectively discard) IP packets on the basis of the value of thesource IP address field in an IP packet. This is because the value ofthe source IP address field in packets that are injected into the packetswitched network 1000 by a network interface 31 may be different thanthe network (e.g. IP) address of the network interface 31. The practiceof such “source address filtering” is not uncommon and is motivated bysecurity concerns associated with source address spoofing. However, anISP may be motivated to selectively turn off source address filtering inorder to support the mechanism disclosed in this invention.Alternatively, signaling or messaging can be employed to cause the ISPto selectively or automatically turn this feature off when aggregationis in use.

A salient feature of performing re-assembly at the network layer is thatthe mechanism is independent of the transport layer protocol used, forexample whether it comprises a TCP, UDP or other such protocol. However,some network operators do not support transit service for IP fragments,and will simply discard IP fragments that are received. Thus it isdesirable to have another mechanism to achieve reassembly at thedestination device.

Upstream Data Transfers with Transport Layer Re-Assembly

If an upstream data transfer request is made over a reliable transportprotocol that supports out-of-order delivery of packets at the networklayer, the reliable transport protocol itself can be used to re-assembletraffic at the ultimate destination device 201. One such reliabletransport protocol is TCP.

With transport layer re-assembly, the aggregation device 201 simplyforwards arriving packets belonging to the same flow for an upstreamdata transfer to either one of the network interfaces 31 or 32,according to any number of load balancing algorithms. The value of thesource network (IP) address field for all such packets, regardless ofwhich network interface is used to carry the packet, is in theillustrated embodiment over-written with the same network (IP) address,which coincides with the network address of one of the networkinterfaces. Thus, it appears to the remote destination device 201 thatthe source of the upstream data transfer is one of the networkinterfaces, either 31 or 32. Due to the different communication links41, 42 that are used to support the upstream data transfers, the packetsoriginally presented at the local port 11 may arrive out of order at theremote destination device 201. However, the transport protocol (e.g.TCP) implemented at the remote destination device will buffer packetsthat arrive out of order, so that data can be presented in the correctorder to the destination application within the remote destinationdevice 201.

Although the TCP protocol is capable of handling packets that arrive outof order, most implementations are optimized for the case where packetsarrive in order with high probability. If packets do arrive at thedestination out of order, this will often cause several acknowledgements(ACKs) being sent back to the source with the same sequence numbervalue. Most implementations of TCP are such that three consecutive ACKswith the same sequence number will enable a “congestion avoidance” mode,whereby the sender will decrease the rate of transmissions. This isbecause in common scenarios of network operation, three consecutive ACKsare an indication of packet loss caused by congestion. With the methoddisclosed herein, however, packets may more commonly arrive at thedestination out of order. This will cause three or more consecutive ACKsto be sent back to the sender even when there is no packet loss withinthe network, and may cause performance degradation if preventativemeasures are not taken.

To remedy this situation, the exemplary aggregation device 101 acts asfollows. Since the same network address, i.e. that of one of the networkinterfaces 31 or 32, is over-written into the source network address ofevery packet in an upstream flow the ACKs will be returned through thenetwork interface whose network address is used as the source networkaddress for all upstream packets in the flow. The aggregation device 101examines the ACKs that are received from the destination device 201through the corresponding network interface. If three or more ACKsarrive from the remote destination device 201 with the same sequencenumber, the third and subsequent ACKs may be discarded by theaggregation device, in an attempt to prevent the source of the datatransfer from receiving three or more ACKs with the same sequence numberand entering a congestion avoidance mode. A timer mechanism may be usedto selectively pass through ACKs with the same sequence number, so thatwhen packet losses and congestion do in fact occur, the upstream datatransfer can proceed in the appropriate way. Other such “filtration” orcontrol mechanisms may be used as well. For example, in another variant,the timing of the receipt of the individual ACKs relative to one anothercan be analyzed to determine if they in fact are related, or constitutea congestion or packet loss condition. This can also be coupled to oneor more parameters associated with the transmission side; i.e., threeACKs would occur when packets arrive substantially out of order, andhence the transmission channel(s) may be the root cause of theout-of-order receipt at the destination. Hence, by monitoring in effect“what goes out when”, the aggregator 101 can in essence haveforeknowledge of or expect the three ACKs when the transmission ishighly asynchronous. It can then selectively desensitize itself (e.g.,filter one or more of the ACKs, set the threshold to more than threeACKs, etc.). Similarly, as previously described, the destination canalso operate dynamically so as to relax its out-of-order processing(including sending multiple ACKs for the same sequence number), relaxits out-of-order standards (e.g., increase jitter buffer size so as toallow greater temporal misalignment), and so forth.

Mobile Devices

The algorithms described herein for control of the aggregation device101 may also be optimized for mobile applications, e.g., so as toconsider particular attributes of the mobile communication links (e.g.,WiFi, cellular, WiMAX, PAN, etc.) over which the data is flowing, aswell as the attributes of the mobile device itself. For example, in onevariant, two different wireless (e.g., WiFi and cellular, two WiFi,etc.) links may asymmetric in their capabilities, security (e.g.,aforementioned 802.1x/RADIUS capability), etc., and hence may be“intelligently” selected for certain types of traffic or times of use.Moreover, since battery conservation on a mobile device is of paramountimportance, the algorithms of the invention may be adapted to optimizepower consumption. For example, in one variant, this optimizationcomprises preferentially using links for the aggregation process thatare more power efficient. Wait or sleep states can also be utilizedwithin the device (and the aggregation algorithm), as can opportunisticor non-real time delivery (e.g., waiting to transmit non-time criticaldata over links when the most power efficient bandwidth becomesavailable).

It is also noted that while described in terms of packet-switchednetwork embodiments, the present invention can also be applied to otherdelivery paradigms, such as e.g., packet-over-circuit (e.g., packet overa PSTN), hybrid networks (such as those employing a gateway between apacket-switched and circuit switched network, such as in exemplaryH.323, H.248 or similar teleconferencing applications, and so forth.

It will be recognized that while certain aspects of the invention aredescribed in terms of a specific sequence of steps of a method, thesedescriptions are only illustrative of the broader methods of theinvention, and may be modified as required by the particularapplication. Certain steps may be rendered unnecessary or optional undercertain circumstances. Additionally, certain steps or functionality maybe added to the disclosed embodiments, or the order of performance oftwo or more steps permuted. All such variations are considered to beencompassed within the invention disclosed and claimed herein.

It will also be appreciated that while described generally in thecontext of business/enterprise or consumer (e.g., residence)applications, the present invention may be readily adapted to othertypes of environments (e.g., government, military, etc.) as well. Myriadother applications are possible.

While the above detailed description has shown, described, and pointedout novel features of the invention as applied to various embodiments,it will be understood that various omissions, substitutions, and changesin the form and details of the device or process illustrated may be madeby those skilled in the art without departing from the invention. Theforegoing description is of the best mode presently contemplated ofcarrying out the invention. This description is in no way meant to belimiting, but rather should be taken as illustrative of the generalprinciples of the invention. The scope of the invention should bedetermined with reference to the claims.

1.-34. (canceled)
 35. An aggregation device comprising: a processingapparatus configured to classify a plurality of packets that arrive viaa first communications interface as belonging to a single communicationsession; the processing apparatus configured to select one of aplurality of network interfaces that will be associated with the singlecommunication session; the processing apparatus configured to forward afirst packet received via the first communications interface to theselected network interface associated with the single communicationsession that the first packet belongs to; the processing apparatusconfigured to update a source network address of the first packet withthe network address associated with the selected network interface; andthe processing apparatus configured to forward a second packet receivedvia the selected network interface for transmission via the firstcommunications interface; wherein a destination network address of thesecond packet is replaced by the source network address.
 36. Theaggregation device of claim 35, wherein the aggregation device isfurther configured to execute a load balancing algorithm, the selectedone of the plurality of network interfaces being selected according tothe load balancing algorithm.
 37. The aggregation device of claim 36,wherein the load balancing algorithm comprises an optimization algorithmconfigured to optimize for higher bandwidth channels.
 38. Theaggregation device of claim 35, wherein the first packet is determinedby the processing apparatus to be of a particular type of data; andwherein the particular type of data is exclusively transferred via theselected network interface.
 39. The aggregation device of claim 36,wherein the load balancing algorithm utilizes a quality of service (QoS)parameter.
 40. The aggregation device of claim 35, wherein the pluralityof network interfaces comprises a cellular network interface and a WiFinetwork interface.
 41. An aggregation device comprising: a processingapparatus configured to receive a request for data via a firstcommunication interface; the processing apparatus configured todetermine an attribute associated with the requested data in order todetermine whether the requested data exceeds a predetermined criterion;and when the requested data exceeds the predetermined criterion, theprocessing apparatus is configured to: generate a plurality of datarequests for portions of the data; send a first subset of the generateddata requests over a first network interface; and send a second subsetof the generated data requests over a second network interface.
 42. Theaggregation device of claim 41, wherein the processing apparatus isfurther configured to: generate additional subsets of the generateddata; and send one or more of the additional subsets over the firstnetwork interface or the second network interface, depending upon thecompletion of the first subset of the generated data requests or thecompletion of the second subset of the generated data requests over thefirst network interface or the second network interface, respectively.43. The aggregation device of claim 42, wherein the first subset of thegenerated data and the second subset of the generated data compriseidentical redundant data requests.
 44. The aggregation device of claim41, wherein the first subset of the generated data and the second subsetof the generated data comprise identical redundant data requests. 45.The aggregation device of claim 44, wherein the processing apparatus isfurther configured to receive the requested redundant data over thefirst network interface and receive the requested redundant data overthe second network interface; and wherein the processing apparatus isfurther configured to discard either the redundant portion received onthe first network interface or the redundant portion received on thesecond network interface depending upon an evaluated criterion.
 46. Theaggregation device of claim 45, wherein the evaluated criterioncomprises a time of arrival for the redundant portion received on thefirst network interface and a time of arrival for the redundant portionreceived on the second network interface.
 47. The aggregation device ofclaim 46, wherein the processing apparatus discards a later arriving oneof the redundant portion received on the first network interface and theredundant portion received on the second network interface.
 48. Theaggregation device of claim 41, wherein the first network interfaceoperates according to a cellular communication protocol and the secondnetwork interface operates according to a WiFi communication protocol.49. The aggregation device of claim 42, wherein the first networkinterface operates according to a cellular communication protocol andthe second network interface operates according to a WiFi communicationprotocol.
 50. The aggregation device of claim 44, wherein the firstnetwork interface operates according to a cellular communicationprotocol and the second network interface operates according to a WiFicommunication protocol.
 51. An aggregation device comprising: aprocessing apparatus configured to receive a plurality of packets via afirst communications interface; the processing apparatus configured toselect one of a plurality of network interfaces associated with thereceived plurality of packets; the processing apparatus configured toforward a first portion of packets received via the first communicationsinterface to the selected one network interface, the first portion ofpackets being modified to match a network layer format requirement forthe selected network interface; the processing apparatus configured toupdate a source network address for the first portion of packets withthe network address associated with the selected network interface; andthe processing apparatus configured to forward a second portion ofpackets received via the selected network interface for transmission viathe first communications interface; wherein a destination networkaddress of the second portion of packets is replaced by the sourcenetwork address and the second portion of packets being modified tomatch a network layer format requirement for the first communicationinterface.
 52. The aggregation device of claim 51, wherein the networklayer format requirement for the first communication interface ispre-stored within a memory of the aggregation device.
 53. Theaggregation device of claim 51, wherein the network layer formatrequirement for the selected network interface is determined by theaggregation device based upon a query of a network interface by theaggregation device.
 54. The aggregation device of claim 51, wherein thenetwork layer format requirement for the selected network interface isdetermined by the aggregation device based upon a query of a destinationdevice over a network.
 55. The aggregation device of claim 51, whereinthe network layer format requirement for the first communicationinterface is determined by aggregation device based upon a query of alocal device via the first communications interface.
 56. The aggregationdevice of claim 51, wherein the network layer format requirement for theselected network interface is a Maximum Transmission Unit (MTU) size.